Virtual Reality System with Head-Mounted Display, Camera and Hand-Held Controllers

ABSTRACT

A method is performed in a virtual-reality system that includes a head-mounted display (HMD), a camera, and a hand-held controller having a plurality of illumination sources to provide light that is detectable by the camera. The method includes displaying, on the HMD, an image of a hand and using the camera to detect movement of the hand-held controller by tracking positions of respective illumination sources of the plurality of illumination sources. The method further includes displaying, on the HMD, motion of the image of the hand corresponding to the detected movement.

RELATED APPLICATION

This application is a continuation of U.S. Nonprovisional applicationSer. No. 14/729,954, filed Jun. 3, 2015, entitled “Virtual RealitySystem with Head-Mounted Display, Camera and Hand-Held Controllers,”which is incorporated by reference in its entirety.

TECHNICAL FIELD

This application relates generally to gaming entertainment andvirtual-reality systems, and more specifically to virtual-realitysystems having an image-capturing device capable of sensing or trackingmovement of hand-held controllers in order to detect a user's handmovements.

BACKGROUND

Gaming entertainment systems typically include a hand-held controller,game controller, or other controller. A user manipulates the controllerto send commands or other instructions to the gaming entertainmentsystem to control a video game or other simulation. For example, thecontroller may be provided with several buttons or knobs operated by theuser, such as a joystick. Each of the buttons or knobs corresponds to adesired action to be carried out on a display of the gamingentertainment or virtual-reality system. Other gaming andvirtual-reality systems further provide virtual-reality gear such as 3Dglasses or mats having motion sensors which the user steps on to trackthe user's feet to give the user a perception of being in virtualreality.

The action carried out on a display or screen of the virtual-reality orgaming system is limited to a representation of a general position ofthe user while on the mat. The user's physical hand motions such aspunching, swinging, or waving and the like are not detectable by themat, therefore these virtual-reality or gaming systems provide only alimited feeling of “reality” to the user.

SUMMARY

Accordingly, there is a need for virtual-reality or gaming systemshaving capability of tracking positions of hand-held controllers tosimulate actual motion of a user holding the controller, therebyenhancing the user virtual-reality experience. This capability may beintegrated with a head-mounted display (HMD), to reduce the number ofcomponents in such systems.

In accordance with some embodiments, a virtual-reality system includes aHMD, a forward-looking camera coupled to the HMD, and a hand-heldcontroller communicatively (e.g., wirelessly) coupleable to the HMD. Thehand-held controller includes a first user-input key, a grip, and anoutward-facing surface coupled to the grip. The hand-held controllerfurther includes a plurality of illumination sources coupled to (e.g.,mounted on or embedded in) the outward-facing surface. The illuminationsources are configured to provide light that is detectable by thecamera.

In some embodiments, the forward-looking camera extends from a frontsurface of the HMD.

In some embodiments, the hand-held controller further includes a cagecoupled to the grip. The outward-facing surface includes an outersurface of the cage. The plurality of illumination sources are coupledto the outer surface of the cage.

In some embodiments, at least a portion of the plurality of illuminationsources are positioned to be detectable to the forward-looking camerawhen the HMD is worn by a user and the user holds the grip in a neutralposition.

In some embodiments, the cage is configured to be positioned above auser's hand when the user holds the grip in the neutral position.

In some embodiments, the hand-held controller further includes auser-input surface that includes the first user-input key.

In some embodiments, the virtual-reality system further includes astructural web coupling the cage to the user-input surface.

In some embodiments, the virtual-reality system further includes asecond user-input key. The second user-input key is a trigger mounted onat least one of the structural web and the grip at a position configuredto be actuated by a middle finger of the user.

In some embodiments, the user-input surface forms an inner front surfaceof the cage.

In some embodiments, the grip is slanted at an angle with respect to theuser-input surface.

In some embodiments, the user-input surface includes a plurality ofuser-input keys including the first user-input key. The user-inputsurface further includes a touch-sensitive surface partitioned into aplurality of sections where each section corresponds to a respectiveuser-input key and includes at least one touch sensor to detect a touchon the corresponding section.

In some embodiments, the grip is integrally formed with the cage.

In some embodiments, the cage is detachably coupled to the grip.

In some embodiments, the plurality of illumination sources includes aplurality of light-emitting diodes (LEDs) and the forward-looking camerais configured to track light emitted by the LEDs.

In some embodiments, the plurality of LEDs includes a plurality ofinfrared LEDs and the forward-looking camera is configured to detectinfrared light emitted by the infrared LEDs.

In some embodiments, the virtual-reality system further includes a firstpower source to supply power to the HMD and the forward-looking camera,and a second power source to supply power to the hand-held controller,including to the plurality of LEDs.

In some embodiments, the plurality of illumination sources includes aplurality of passive reflectors, and the camera includes an illuminationsource to provide light to the passive reflectors. The camera may alsoinclude a sensor to detect light reflected back to the camera by thepassive reflectors.

In some embodiments, the first user-input key is selected from the groupconsisting of a thumbstick, a button, a trigger, and a directional pad.The button is selected from the group consisting of an A or X button, aB or Y button, a start button, a back button, a forward button, and ahome button.

In some embodiments, the forward-looking camera is external to andelectrically coupleable to the HMD.

In accordance with some embodiments, a virtual-reality system includes ahead-mounted display (HMD), a forward-looking camera coupled to the HMD,and two hand-held controllers. Each of the hand-held controllers iscommunicatively (e.g., wirelessly) coupleable to the HMD and includes arespective user-input key, a respective grip, a respectiveoutward-facing surface coupled to the respective grip, and a respectiveplurality of illumination sources coupled to the respectiveoutward-facing surface and configured to provide light that isdetectable by the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIGS. 1A-1C illustrate exemplary virtual-reality systems including afront-looking camera extending from a head-mounted display (HMD) inaccordance with some embodiments.

FIG. 2 illustrates an exemplary virtual-reality system including ahead-mounted display and an external camera in accordance with someembodiments.

FIG. 3A and FIG. 3B illustrate isometric views of a hand-held controllerof the exemplary virtual-reality system in accordance with someembodiments.

FIG. 4 is a block diagram illustrating an electrical configuration ofthe exemplary virtual-reality system in accordance with someembodiments.

FIG. 5A is a block diagram of an exemplary head-mounted display inaccordance with some embodiments.

FIG. 5B is a block diagram of an exemplary camera in accordance withsome embodiments.

DETAILED DESCRIPTION

Virtual-reality or gaming systems may include hand-held controllers heldin one or both hands by a user while playing a video game or carryingout some other virtual reality activity in order to operate theuser-input keys (e.g., buttons, thumbstick, directional pad, trigger,etc.) on the controller. While playing the game or carrying out thevirtual-reality activity, the user may become so immersed in the game asto move their hands in a manner mimicking a desired action (e.g.,performed by pressing one of the buttons while holding the controller).For example, during playing a boxing game a user may press an “L” buttoncorresponding to a left hand punch while simultaneously jerking theirleft hand for a more “real” sensation. It is desirable to display thehand motion of the user as a corresponding motion by an image subject onthe screen in the virtual reality system.

Accordingly, the present invention describes virtual reality systemscapable of detecting physical positions of hand-held controllers atvarious points in time to simulate actual hand movements of usersholding the controllers to allow easy tracking of the user handmovements and enhance the user virtual-reality experience. The detectedpositions and movements of the hand-held controllers may be used asadditional commands to control various aspects of the game or othersimulation being played.

In some embodiments a virtual-reality system includes a head-mounteddisplay (HMD), a forward-looking camera coupled to the HMD, and ahand-held controller communicatively coupleable to the HMD (e.g., suchthat it can communicate wirelessly with the HMD). The hand-heldcontroller includes one or more user-input keys, a grip, anoutward-facing surface coupled to the grip, and illumination sourcescoupled to (e.g., mounted on or embedded in) the outward-facing surfaceof the controller. The forward-looking camera is adapted to detect lightreflected or emitted by the illumination sources. In some embodiments,the forward-looking camera is directly coupled to and extends from afront surface of the HMD. In other embodiments, the forward-lookingcamera is an external camera communicatively coupled to, but physicallypositioned separate from the HMD. The HMD uses the forward-lookingcamera to detect movement of the illumination sources by trackingpositions of light reflected or emitted in order to model actions of theuser's hand(s) in the virtual-reality system. The camera may detectvarious movements of the hand-held controller, such as punchingmovements, throwing movements, hitting movements when playing a sport,and the like. Motions of the hand-held controller correspond to variouscommands such that the motions are transferred into actions in thevirtual reality system.

Reference will now be made to embodiments, examples of which areillustrated in the accompanying drawings. In the following description,numerous specific details are set forth in order to provide anunderstanding of the various described embodiments. However, it will beapparent to one of ordinary skill in the art that the various describedembodiments may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, circuits, andnetworks have not been described in detail so as not to unnecessarilyobscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are used onlyto distinguish one element from another. For example, a first user-inputkey could be termed a second user-input key, and, similarly, a seconduser-input key could be termed a first user-input key, without departingfrom the scope of the various described embodiments. The firstuser-input key and the second user-input key are both user-input keys,but they are not the same user-input key.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof. The term “exemplary” is used herein in the senseof “serving as an example, instance, or illustration” and not in thesense of “representing the best of its kind.”

FIGS. 1A and 1B illustrate exemplary virtual reality systems 100A and100B including a front-looking camera 102 coupled to (e.g., extendingfrom) a head-mounted display (HMD) 105, and a hand-held controller 300in accordance with some embodiments. The hand-held controller 300generally comprises a first user-input key 120A, a grip 130, anoutward-facing surface 145 coupled to the grip 130, and a plurality ofillumination sources 150 coupled to (e.g., mounted on or embedded in)the outward-facing surface 145. The illumination sources 150 areconfigured to emit or reflect light that is detectable by camera 102.The camera 102 is positioned to detect movement in positions of theillumination sources 150 when the user makes a motion (e.g., waving,swinging, punching, shaking, or any other hand motion) while holding thegrip 130 of the controller. By detecting a movement of the illuminationsources 150 on the controller 300, the camera is able to capturesequential positions of the controller 300 over time, and thus allowmotion of an image subject to be modeled in virtual reality based onactual physical motions made by the user (e.g., punching, swinging,etc.) as described above. The HMD 105 is configured to display a viewwhich shifts as a user shifts their head in a corresponding direction ortilts their head at an angle. The HMD 105 is communicatively coupleableto the controller 300 and the camera 102 so as to be able to display thedetected motions of the controller 300 along with motions of the head ofthe user as those of an image subject of the virtual reality system 100.For example, the HMD 105 communicates wirelessly with the controller 300and communicates with the camera 102 through a wired or wirelessconnection.

In some embodiments, as illustrated for the virtual-reality system 100A(FIG. 1A), the camera 102 extends from (e.g., is mounted to) the top ofthe HMD 105 (e.g., at a downward facing angle) such that the controller300 having the plurality of illumination sources 150 is within view ofthe camera 102 when the user holds the grip 130 in a neutral position.Alternatively, the camera 102 extends from (e.g., is mounted to) a sideof the HMD 105 or the bottom of the HMD 105. In these examples, theforward-looking camera 102 is external to and communicatively coupled orcoupleable to the HMD 105 (e.g., through a wired connection with the HMD105 or wirelessly). A neutral position refers to when users hold thecontroller 300 in front of them with the grip 130 between palm andfingers and otherwise relax their arms and wrists. The distance at whichthe camera 102 is extendable from the HMD 105 may be adjustabledepending on how far away the user holds the controller 300 from thetheir body.

In some embodiments, as illustrated for the virtual-reality system 100B(FIG. 1B), the camera 102 extends from a front surface of the HMD 105.For example, a first portion of the camera 102 is in front of the HMD105 while a second portion of the camera 102 is housed within the HMD105. Extending the camera 102 from the front surface of the HMD 105provides a wide (e.g.,)180° field of view for the camera 102. In stillother embodiments, the camera is embedded entirely within the HMD 105(e.g., with the lens exposed through a transparent portion of the frontsurface of the HMD 105).

The camera 102 may be configured with sensors to sense light emitted orreflected by the illumination sources 150. The camera 102 is configuredto sense a position of the controller 300 or illumination sources 150based on the light emitted. The camera 102 or HMD 105 thus may determinethe position and orientation of the controller 300. In some embodiments,based on a distance of the controller 300 from the user's body, thecamera 102 is adapted to either extend away from or retract towards theHMD. In some embodiments, an angle at which the camera faces thecontroller 102 is also similarly adjustable depending on orientation ofthe controller 300 and the plurality of illumination sources.

In some embodiments, forward facing camera 102 is positioned to detectthe light emitted or reflected by the illumination sources 150 when theoutward-facing surface 145 is positioned above the user's hand in theneutral position.

FIG. 2 illustrates an exemplary virtual-reality system 200 including ahead-mounted display 105 and an external camera 202 in accordance withsome embodiments. The camera 202 is external to and communicatively(e.g., wirelessly) coupleable to the HMD 105. Here, the camera 202 ispositioned in front of the user and at least a portion of theillumination sources 150 are positioned to face away from the user so asto be visible or detectable to the external camera 202 when the userholds the grip 130 in the neutral position.

As described for the camera 102, the camera 202 detects movement of theillumination sources 150 when the user makes a motion (e.g., waving,swinging, punching, shaking, or any other hand motion) while holding thegrip 130 of the controller. In some embodiments, as illustrated in FIG.2, the outward-facing surface 145 is positioned such that it is locatedabove the user hand when the user holds the grip 130 in the neutralposition. Given this orientation, the outward-facing surface 145 iswithin the view of the external forward-looking camera 202 which isseparate from the HMD 105.

In some embodiments, the camera 102, 202 may be customized for thepurposes of the virtual reality system 100. In some embodiments, thecamera 102, 202 may include but not be limited to a commerciallyavailable camera product of any type or brand, such as for example aweb-cam, an IR capable camera, and/or a USB camera, etc.

In some embodiments, the camera 102, 202 is electrically connected to apower source which may or may not be the same power source providingpower to the HMD 105. The camera 102, 202 and the HMD 105 may bewireless; therefore, the power source may be one or more batteries.

In some embodiments, the illumination sources 150 are positioned toallow the camera 102, 202 to detect at least six degrees of freedom ofthe controller 102. The six degrees of freedom are the controller's 300position within x, y and z coordinates of space and the controller's 300orientation—which includes the controller's 300 yaw, pitch and roll. Thesix degrees of freedom detected by the camera 102 are used to determinethe user's movements of the controller 300 and model these movements invirtual reality.

In some embodiments, the illumination sources 150 are light-emittingdiodes (LEDs). In some embodiments, the LEDs 150 are infrared (IR) LEDs.The camera 102 is configured to detect the IR light emitted by the IRLEDs on the controller 300 and record the signals sensed from theemitted light to determine the position and orientation of thecontroller 300.

In some embodiments, the LEDs may be positioned on the outward-facingsurface 145 of the controller 300 in any suitable pattern, order, orarray. In some embodiments, the outward-facing surface 145 is an outersurface of a cage 140 coupled to the grip 130. The LEDs may be fixedlyor detachably positioned on, and thus coupled to, the cage 140 by anyappropriate method. For example, the LED's may be mounted on or embeddedwithin the outer surface 145 of the cage 140. Alternatively, the LEDsmay be on a sleeve that surrounds the cage 140 and effectively forms theouter surface 145 of the cage 140. Although the LEDs are described asbeing positioned on the outer 145 surface of the cage 140, they mayadditionally or alternatively be coupled to any other surface on thecage 140 and/or the rest of the controller 300.

The LEDs are electrically connected to a power source which may or maynot be same power source providing power to the controller 300. Thecontroller 300 may be wireless; therefore, the power source may be oneor more batteries. The LEDs may be housed in diffused cases including acurrent limiting resistor to keep the current from the power source tothe LED below the LED's maximum current rating so as to ensure maximumlife of the LEDs. The LEDs may be activated when a suitable voltage isapplied. By virtue of the LEDs being positioned in an area on thecontroller 300 detectable to the camera 102, 202, motion of the lightproduced by the LEDs that is detected by the camera 102, 202 is used asan indication of the positions and motion of the controller 300. In thisway, motion of the controller is tracked by the camera 102, 202,allowing for corresponding virtual-reality hand motions to be shown onthe HMD 105. For example, when the user makes a punching motion whileplaying a boxing game, movement of the LEDs in a manner corresponding toa punch may be detected and used to model the user's motion for theimage subject displayed on the HMD 105 in the virtual reality system100. In this way, the present invention provides the advantage ofenhancing the virtual reality experience for the user by adding theadditional element of hand motions to the image subject.

In some embodiments, the illumination sources 150 are passivereflectors. In some embodiments, the camera 102, 202 is adapted toinclude an illumination source (e.g., a flash) and to provide light tothe passive reflectors. For example, FIG. 1C shows a virtual-realitysystem 100C in which the camera 102 includes an illumination source(e.g., a flash) 160. (The illumination source may be directly connectedto the rest of the camera 102 or may be separate from the rest of thecamera 102. The illumination source may be considered part of the camera102 or may be considered a separate component of the virtual-realitysystem.) The illumination source 160 may be integrated into the HMD 160(e.g., such that it provides illumination through a transparent portionof the front surface of the HMD 160), mounted on the HMD 160 (e.g., onthe top, bottom or side of the HMD 160) or separate from the HMD 160. Inthe example of FIG. 2, an illumination source may be integrated in thecamera 202, mounted on the camera 202, or separate from the camera 202.The passive reflectors receive light (e.g., from the illumination source160) and reflect the light back in the direction of the camera 102, 202.The camera 102. 202 includes a sensor to detect the light reflected backby the passive reflectors and record the signals sensed from thereflected light to determine the position and orientation of thecontroller 300.

In some embodiments, the controller 300 includes the first user-inputkey 120A and may include one or more additional user-input keys 120B,120C. A user-input key is a button, knob, switch, thumbstick,directional pad, or any other such part that a user presses ormanipulates in some other way to carry out a specific action in avirtual reality system (e.g., during gaming).

FIG. 3A and FIG. 3B illustrate isometric views of a hand-held controller300 of the exemplary virtual reality system 100 in accordance with someembodiments. In some embodiments, the first user-input key 120A may beselected from the group consisting of a thumbstick, a button, a trigger,and a directional pad. In some embodiments, the first user-input key120A may be the button selected from the group consisting of an A or Xbutton, a B or Y button, a start button, a back button, a forwardbutton, and a home button. The A or B buttons may correspond to aselection action between at least two choices presented to the user inthe gaming system. The X or Y button may correspond to a negative oraffirmative decision to be made by the user dictating how the imagesubject will proceed in the game. X may correspond to an action of “NO”or “END” and Y may correspond to “YES” or “PROCEED/CONTINUE.” The startbutton may be a button activated by the user to begin thevirtual-reality (e.g., gaming) experience, and the back and forwardbuttons may indicate a direction in which the user desires the imagesubject to move. The home button may be a button activated to return thegaming experience back to a main menu or to start the game or activityfrom the beginning.

In some embodiments, the hand-held controller 300 further comprises auser-input surface 110 that includes the first user-input key 120A. Theuser-input surface 110 includes a plurality of user-input keys 120A,120B and 120C. Alternatively, the user-input surface includes a singleuser-input key. In the example of FIGS. 1A-1C, FIG. 2, and FIGS. 3A and3B, the user input keys include a thumbstick 120A and buttons 120B and120C. Thus, the user-input surface 110 is a surface on the controller300 where the user delivers an input by activating one or moreuser-input keys (e.g., by pressing a button or pushing a knob)corresponding to an action that the user desires to carry out in thevirtual-reality system 100 (e.g., the virtual reality system 100A, 100B,or 100C).

Each of the user-input keys 120A, 120B and 120C is configured tocommunicate with the virtual-reality system 100 so as to translate anoperation of the corresponding user-input key by the user into acorresponding action of the image subject displayed on the HMD 105 ofthe virtual reality system 100.

In some embodiments, the user-input surface 110 includes a plurality ofuser-input keys including the first user-input key 120A, and respectiveuser-input keys 120B, 120C of the plurality of user-input keys areselected from a group consisting of a thumbstick, a button, a trigger,and a directional pad.

In some embodiments, a home button is positioned further away from theother user-input keys. This configuration would allow for user-inputkeys that are used most (e.g. a directional pad used to dictate adirection of movement of the image subject, e.g., up-down-left-right) tobe placed closer to the vicinity of the fingers and thumb. Thisconfiguration provides the advantage that the user would need to reachless to press the more frequently used user-input keys, therebymitigating the possibility of ergonomic ailments associated withoverreaching and overstretching fingers.

In some embodiments, at least a portion of the user-input surface 110 isa touch-sensitive surface partitioned into a plurality of sections. Eachsection corresponds to a respective user-input key of the plurality ofuser-input keys. In this configuration, at least one touch sensor ispositioned on a bottom surface of the user-input surface 110, asillustrated in FIG. 4, to detect a touch on the corresponding section.When a touch of a key by the user is detected by the sensors, the actionassociated with the corresponding user-input key touched is translatedto an action of the subject image in the virtual reality system.

In some embodiments, the grip 130 is coupled to the user-input surface110. The grip 130 is a protruding structure of the controller 300 whichthe user grips in one hand to hold the controller 300. Thisconfiguration allows for the user to be able to grip the controller 300between a palm and fingers (e.g., three or less fingers) while freeingup the thumb and, in some embodiments, another finger (e.g. the middlefinger), for operating the user-input keys 120A, 120B and 120C. In someembodiments, the middle finger is freed to operate a trigger 190 mountedat least in part on the grip 130 as shall be described below.

In some embodiments the grip 130 is a separate part of the controller300 that is removably coupled to the user input surface 110 and/or cage140. The grip 130 and the user-input surface may be coupled by a methodappropriate for their materials of construction. For example, the gripand user-input surface 110 may be formed of a hard plastic and may becoupled to each other by ultrasonic welding. Alternatively, the grip 130and the user-input surface 110 may be coupled to each other by afastening mechanism such as a screw or a bolt, or may be threadedlyengaged with each other.

In some embodiments, the grip 130 is integrally formed with theuser-input surface 110 and/or the cage 140, as one part (e.g., which maybe formed from molding).

In some embodiments, the grip 130 is slanted at a predetermined anglewith respect to the user-input surface 110 (e.g., with a plane throughthe user-input surface or a portion thereof) in order to provide acomfortable (e.g., optimum) ergonomic balance for a user between holdingthe grip in and using a thumb to operate the at least one user-inputkey.

In some embodiments, the cage 140 is coupled to the user-input surface110. The cage 140, which may also be referred to as a tracking cage, hasthe outer surface 145 on which the plurality of illumination sources 150is positioned.

In the example of FIGS. 1A-1C and 3A-3B, the user-input surface 110 isoutward-facing with respect to the cage 140. Alternatively, theuser-input surface 110 may be inward-facing with respect to the cage140, as illustrated in FIG. 2. For example, in some embodiments theuser-input surface 110 forms an inner front surface of the cage 140 oris contiguous with the inner surface of the cage 140.

In some embodiments, the grip 130 and/or cage 140 may be formed of anover-molded rubber material (e.g., so as to provide a surface providingsufficient friction with a user's palm thus improving the grip). In someembodiments, the grip 130 and/or cage 140 may be formed of a hardplastic, including, but not limited to high density polyethyleneproviding increased rigidity in structure. Additionally, any othersuitable materials may be used.

In some embodiments, the cage 140 may be detachably coupled to at leastone of the user-input surface 110 and the grip 130. The cage 140 may beslidably coupled to the user-input surface 110 through a protrusionspanning a width of each end portion of the cage 140 being slidablyengaged with a corresponding groove positioned on an outer circumferenceof the user-input surface 110. The cage 140 may be coupled to the grip130 through a fastening mechanism such as a bolt, a screw or the like.The detachable configuration of the cage 140 to the grip 130 or theuser-input surface 110 yields the advantage of separating theaforementioned components for calibration as necessary. Detachablecoupling of the components also allows for a separate and potentiallycheaper manufacturing process of the parts. Furthermore, detachablecoupling of the cage 140 to at least one of the user-input surface 110and the grip 130 allows for separation thereof upon dropping of thecontroller 300, thereby reducing the need to replace the entire unitupon damage, but instead focus on fixing/replacing the separate damagedpart.

In some embodiments, as illustrated in FIGS. 3A and 3B, the controller300 may further comprise a structural web 195 coupling the cage 140 tothe user-input surface 110. The large structural web 195 providesfurther rigidity in structure to the coupling between the cage 140 andthe user-input surface 110 to mitigate damage and separation of thesecomponents upon dropping of the controller 300 by the user.

In some embodiments, a trigger 190 is mounted at least in part on thestructural web 195. That is, the trigger 190 may be mounted between thestructural web 190 and the grip 130. This configuration yields theadvantage that the trigger is positioned adjacent to a location of auser's finger (e.g., middle finger) when the grip 130 is held in theneutral position. In some embodiments, the trigger may be both pushedand pulled by the middle finger, thus providing increased control inmanipulating the trigger to achieve a desired action. The trigger 190 isan example of a user-input key.

FIG. 4 is a block diagram illustrating an electrical configuration of anexemplary hand-held controller 300 in accordance with some embodiments.The hand-held controller 300 includes an input board 402 and a mainboard 403 coupled to the input board 402. The input board 402 includes atrigger motion sensing device 425, a thumbstick 430, buttons 435, and acapacitive touch controller 440. In other examples, the input board 402may include additional or alternative user-input keys. The triggermotion sensing device 425 detects user activation of a trigger (e.g.,trigger 190).

The capacitive touch controller 440 is coupled to multiple sensors suchthat the input board 402 receives sensed signals from capacitive sensorsresulting from a user's touch. For example, the capacitive sensorsinclude a trigger sensor 405, a thumbstick sensor 410, an “A” buttonsensor 415, and/or a “B” button sensor 420. For example, the triggersensor 405 may sense when a user touches the trigger. Similarly, thethumbstick sensor 410 senses a signal resulting from the user touchingthe thumbstick 410. Further, the button sensors 415 and 420 sensesignals resulting from the user touching the buttons 415 and 420. Othercapacitive sensors may be included for other user-input keys (e.g., adirectional pad).

The mainboard 403 includes a controller 460, a haptics driver 465, amiddle finger position sensor 475, power path 490, motion trackingsensors/processors 495 and an illumination source driver 480. Thehaptics driver 465 drives a haptics output device 470 that provideshaptic effects. An example of the haptics output device 470 includes ashort vibration feedback device that, when activated, causes thehand-held controller 300 to vibrate.

The mainboard 403 may be coupled to an antenna 445 to wirelessly receiveand transmit signals. The hand-held controller (e.g., controller 300)thus may be wireless. The mainboard 403 may also be coupled to a powersource (e.g., a battery 450) to provide power supply to the hand-heldcontroller. The power may be supplied to the mainboard 403, and also tothe input board 402, through a power path 490.

The illumination source driver 480 (e.g., LED driver) drivesillumination sources 485 (e.g., LEDs on the outer surface of the cage140) under the control of the controller 460, and thus turns theillumination sources 485 on or off.

The middle finger position sensor 475 senses a position of the middlefinger (e.g. when a user activates the trigger 190) and this informationis processed by the controller 460. The motion trackingsensors/processors 495 include a plurality of motion sensors (e.g.accelerometers and/or gyroscopes) which tracks motion of the controllerbased on motions made by the user.

The camera 102, 202 includes at least one sensor for sensing lightemitted or reflected from the plurality of illumination sources (e.g.,LEDs or passive reflectors) and a controller for processing the lightimages received for the illumination sources 485 to detect positions ofthe controller over time.

In some embodiments, the HMD includes at least one sensor to sensesignals from the controller 460 and a controller to process thesesignals into images displayed on the HMD. The processor of the HMD mayfurther be configured to process information received from the camera102, 202 relating to positions of the hand-held controller 300 fordisplay on the HMD.

FIG. 5A is a block diagram of an exemplary HMD 105 in accordance withsome embodiments. The HMD 105 includes an HMD 502, one or more sensors504, a controller 506, and a camera 102. The controller 506 is coupledto an antenna 510 for wireless communication (e.g., with hand-heldcontrollers 300). The HMD 105 (e.g., including the camera 102) ispowered by a battery 512.

FIG. 5B is a block diagram of an exemplary camera 530 in accordance withsome embodiments. The camera 530 is an example of a camera 102 or 202.The camera 530 includes sensor(s) 532, a controller 534, and an optionalillumination source (e.g., flash) 536. The controller 534 is coupled toan antenna 538 for wireless communication (e.g., with an HMD 105 and/orhand-held controllers 300). The camera 530 is powered by a battery 540;alternatively, the camera 530 is powered by a battery it shares with theHMD 105 (e.g., battery 512, FIG. 5A).

Some embodiments are directed to two hand-held controllers 300—one to beheld in each of a user's hands. In some embodiments, the two controllers300 may be identical, but for a position of at least one of theuser-input keys, so as to be adapted specifically for either a left orright hand in which the controller is to be held. The first controllerthus may be a right-handed controller and the second controller may be aleft-handed controller. In other embodiments, the two controllers may beagnostic with respect to handedness (e.g., with both controllers havingthe same configuration of user-input keys, or with one controller havinga configuration of user-input keys different than the other).

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the scope of the claims to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The embodiments were chosen in order to best explain theprinciples underlying the claims and their practical applications, tothereby enable others skilled in the art to best use the embodimentswith various modifications as are suited to the particular usescontemplated.

What is claimed is:
 1. A method comprising, in a virtual-reality systemcomprising a head-mounted display (HMD), a camera, and a hand-heldcontroller having a plurality of illumination sources to provide lightthat is detectable by the camera: displaying, on the HMD, an image of ahand; using the camera to detect movement of the hand-held controller,comprising tracking positions of respective illumination sources of theplurality of illumination sources; and displaying, on the HMD, motion ofthe image of the hand corresponding to the detected movement.
 2. Themethod of claim 1, wherein: the detected movement of the hand-heldcontroller results from a user waving a hand holding the hand-heldcontroller while the user is wearing the HMD; and displaying the motionof the image of the hand comprises showing the image of the hand wavingon the HMD.
 3. The method of claim 1, wherein: the detected movement ofthe hand-held controller results from a user swinging an armcorresponding to a hand holding the hand-held controller while the useris wearing the HMD; and displaying the motion of the image of the handcomprises showing the image of the hand moving in accordance with theswinging.
 4. The method of claim 1, wherein: the detected movement ofthe hand-held controller results from a user punching with a handholding the hand-held controller while the user is wearing the HMD; anddisplaying the motion of the image of the hand comprises showing theimage of the hand throwing a punch.
 5. The method of claim 1, wherein:the detected movement of the hand-held controller results from shakingof a user's hand holding the hand-held controller while the user iswearing the HMD; and displaying the motion of the image of the handcomprises showing the image of the hand shaking.
 6. The method of claim1, wherein the camera is mounted on the HMD.
 7. The method of claim 1,wherein the camera is external to the HMD and communicatively coupled tothe HMD.
 8. The method of claim 1, wherein: the plurality ofillumination sources comprises a plurality of LEDs; and the trackingcomprises tracking positions of light emitted by respective LEDs of theplurality of LEDs.
 9. The method of claim 1, wherein: the plurality ofillumination sources comprises a plurality of passive reflectors; andthe tracking comprises tracking positions of light reflected byrespective passive reflectors of the plurality of passive reflectors.10. The method of claim 9, wherein: the camera comprises an illuminationsource; and the method further comprises using the illumination sourceto provide light to the passive reflectors.
 11. The method of claim 1,wherein: the detected movement of the hand-held controller correspondsto a command in a virtual-reality simulation; and displaying the motionof the image of the hand comprises showing an action corresponding tothe command.